---

title: MultiInputNet


keywords: fastai
sidebar: home_sidebar

summary: "This is an implementation created by Ignacio Oguiza - timeseriesAI@gmail.com. It can be used to combine different types of deep learning models into a single one that will accept multiple inputs from a MixedDataLoaders."
description: "This is an implementation created by Ignacio Oguiza - timeseriesAI@gmail.com. It can be used to combine different types of deep learning models into a single one that will accept multiple inputs from a MixedDataLoaders."
nb_path: "nbs/130_models.MultiInputNet.ipynb"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/130_models.MultiInputNet.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="MultiInputNet" class="doc_header"><code>class</code> <code>MultiInputNet</code><a href="https://github.com/timeseriesAI/tsai/tree/main/tsai/models/MultiInputNet.py#L11" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>MultiInputNet</code>(<strong>*<code>models</code></strong>, <strong><code>c_out</code></strong>=<em><code>None</code></em>, <strong><code>reshape_fn</code></strong>=<em><code>None</code></em>, <strong><code>multi_output</code></strong>=<em><code>False</code></em>, <strong><code>custom_head</code></strong>=<em><code>None</code></em>, <strong><code>device</code></strong>=<em><code>None</code></em>, <strong>**<code>kwargs</code></strong>) :: <code>Module</code></p>
</blockquote>
<p>Same as <code>nn.Module</code>, but no need for subclasses to call <code>super().__init__</code></p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">fastai.data.transforms</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.data.all</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.models.utils</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.models.InceptionTimePlus</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.models.TabModel</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.learner</span> <span class="kn">import</span> <span class="o">*</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">dsid</span> <span class="o">=</span> <span class="s1">&#39;NATOPS&#39;</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">splits</span> <span class="o">=</span> <span class="n">get_UCR_data</span><span class="p">(</span><span class="n">dsid</span><span class="p">,</span> <span class="n">split_data</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">ts_features_df</span> <span class="o">=</span> <span class="n">get_ts_features</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stderr output_text">
<pre>Feature Extraction: 100%|██████████| 40/40 [00:07&lt;00:00,  5.08it/s]
</pre>
</div>
</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">tfms</span>  <span class="o">=</span> <span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="p">[</span><span class="n">Categorize</span><span class="p">()]]</span>
<span class="n">batch_tfms</span> <span class="o">=</span> <span class="n">TSStandardize</span><span class="p">()</span>
<span class="n">ts_dls</span> <span class="o">=</span> <span class="n">get_ts_dls</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">splits</span><span class="o">=</span><span class="n">splits</span><span class="p">,</span> <span class="n">tfms</span><span class="o">=</span><span class="n">tfms</span><span class="p">,</span> <span class="n">batch_tfms</span><span class="o">=</span><span class="n">batch_tfms</span><span class="p">)</span>
<span class="n">ts_model</span> <span class="o">=</span> <span class="n">build_ts_model</span><span class="p">(</span><span class="n">InceptionTimePlus</span><span class="p">,</span> <span class="n">dls</span><span class="o">=</span><span class="n">ts_dls</span><span class="p">)</span>

<span class="c1"># ts features</span>
<span class="n">cat_names</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">cont_names</span> <span class="o">=</span> <span class="n">ts_features_df</span><span class="o">.</span><span class="n">columns</span><span class="p">[:</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span>
<span class="n">y_names</span> <span class="o">=</span> <span class="s1">&#39;target&#39;</span>
<span class="n">tab_dls</span> <span class="o">=</span> <span class="n">get_tabular_dls</span><span class="p">(</span><span class="n">ts_features_df</span><span class="p">,</span> <span class="n">cat_names</span><span class="o">=</span><span class="n">cat_names</span><span class="p">,</span> <span class="n">cont_names</span><span class="o">=</span><span class="n">cont_names</span><span class="p">,</span> <span class="n">y_names</span><span class="o">=</span><span class="n">y_names</span><span class="p">,</span> <span class="n">splits</span><span class="o">=</span><span class="n">splits</span><span class="p">)</span>
<span class="n">tab_model</span> <span class="o">=</span> <span class="n">build_tabular_model</span><span class="p">(</span><span class="n">TabModel</span><span class="p">,</span> <span class="n">dls</span><span class="o">=</span><span class="n">tab_dls</span><span class="p">)</span>

<span class="c1"># mixed</span>
<span class="n">mixed_dls</span> <span class="o">=</span> <span class="n">get_mixed_dls</span><span class="p">(</span><span class="n">ts_dls</span><span class="p">,</span> <span class="n">tab_dls</span><span class="p">)</span>
<span class="n">MultiModalNet</span> <span class="o">=</span> <span class="n">MultiInputNet</span><span class="p">(</span><span class="n">ts_model</span><span class="p">,</span> <span class="n">tab_model</span><span class="p">)</span>
<span class="n">learn</span> <span class="o">=</span> <span class="n">Learner</span><span class="p">(</span><span class="n">mixed_dls</span><span class="p">,</span> <span class="n">MultiModalNet</span><span class="p">,</span> <span class="n">metrics</span><span class="o">=</span><span class="p">[</span><span class="n">accuracy</span><span class="p">,</span> <span class="n">RocAuc</span><span class="p">()])</span>
<span class="n">learn</span><span class="o">.</span><span class="n">fit_one_cycle</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mf">1e-3</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea ">
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: left;">
      <th>epoch</th>
      <th>train_loss</th>
      <th>valid_loss</th>
      <th>accuracy</th>
      <th>roc_auc_score</th>
      <th>time</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>0</td>
      <td>1.755242</td>
      <td>1.587535</td>
      <td>0.461111</td>
      <td>0.910037</td>
      <td>00:05</td>
    </tr>
  </tbody>
</table>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="p">(</span><span class="n">ts</span><span class="p">,</span> <span class="p">(</span><span class="n">cat</span><span class="p">,</span> <span class="n">cont</span><span class="p">)),</span><span class="n">yb</span> <span class="o">=</span> <span class="n">mixed_dls</span><span class="o">.</span><span class="n">one_batch</span><span class="p">()</span>
<span class="n">learn</span><span class="o">.</span><span class="n">model</span><span class="p">((</span><span class="n">ts</span><span class="p">,</span> <span class="p">(</span><span class="n">cat</span><span class="p">,</span> <span class="n">cont</span><span class="p">)))</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>torch.Size([64, 6])</pre>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">tab_dls</span><span class="o">.</span><span class="n">c</span><span class="p">,</span> <span class="n">ts_dls</span><span class="o">.</span><span class="n">c</span><span class="p">,</span> <span class="n">ts_dls</span><span class="o">.</span><span class="n">cat</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>(6, 6, True)</pre>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">learn</span><span class="o">.</span><span class="n">loss_func</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">



<div class="output_text output_subarea output_execute_result">
<pre>FlattenedLoss of CrossEntropyLoss()</pre>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

</div>
 

